Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 483 encoding errors #484

Open
wants to merge 7 commits into
base: release/2.1.2
Choose a base branch
from

Conversation

nigelmegitt
Copy link
Collaborator

Fix #483 by using codecs.open and ignoring encoding errors. Seems a bit scary/naughty somehow - I would prefer to fix it upstream if I knew the cause.

No longer needed because it has been merged into the core.
Track a change to proxy handling in Autobahn / twisted for #477. There's probably a more elegant way, but this works for now... Lacking a test, but have verified it locally.
We don't need the HTTPProxyConfig object at all anymore, just a `dict` will do.
Have seen this error a couple of times during live operation:

```Shell
Unhandled error in Deferred:
[CRITICAL] (2018-05-08 13:43:42,705) in twisted[154] - Unhandled error in Deferred:

[CRITICAL] (2018-05-08 13:43:42,705) in twisted[154] -
Traceback (most recent call last):
  File "/Users/megitn02/Code/ebu/ebu-tt-live-toolkit/venv/lib/python2.7/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
    result = f(*args, **kw)
  File "/Users/megitn02/Code/ebu/ebu-tt-live-toolkit/ebu_tt_live/node/consumer.py", line 134, in convert_next_segment
    end=self.last_segment_end + self._segment_length
  File "/Users/megitn02/Code/ebu/ebu-tt-live-toolkit/ebu_tt_live/node/consumer.py", line 122, in get_segment
    sequence_number=self._segment_counter
  File "/Users/megitn02/Code/ebu/ebu-tt-live-toolkit/ebu_tt_live/documents/ebutt3.py", line 1004, in extract_segment
    document_segments=document_segments
  File "/Users/megitn02/Code/ebu/ebu-tt-live-toolkit/ebu_tt_live/documents/ebutt3_splicer.py", line 29, in __init__
    self._do_splice()
  File "/Users/megitn02/Code/ebu/ebu-tt-live-toolkit/ebu_tt_live/documents/ebutt3_splicer.py", line 66, in _do_splice
    merged_body = merged_body.merge(current_tt.body, self._dataset)
  File "/Users/megitn02/Code/ebu/ebu-tt-live-toolkit/ebu_tt_live/bindings/__init__.py", line 974, in merge
    self._merge_deconflict_ids(element=other_elem, dest=merged_body, ids=ids)
  File "/Users/megitn02/Code/ebu/ebu-tt-live-toolkit/ebu_tt_live/bindings/__init__.py", line 949, in _merge_deconflict_ids
    cls._merge_deconflict_ids(item.value, copied_elem, ids)
  File "/Users/megitn02/Code/ebu/ebu-tt-live-toolkit/ebu_tt_live/bindings/__init__.py", line 949, in _merge_deconflict_ids
    cls._merge_deconflict_ids(item.value, copied_elem, ids)
  File "/Users/megitn02/Code/ebu/ebu-tt-live-toolkit/ebu_tt_live/bindings/__init__.py", line 949, in _merge_deconflict_ids
    cls._merge_deconflict_ids(item.value, copied_elem, ids)
  File "/Users/megitn02/Code/ebu/ebu-tt-live-toolkit/ebu_tt_live/bindings/__init__.py", line 942, in _merge_deconflict_ids
    log.debug('processing child: {} of {}'.format(item.value, element))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in position 1: ordinal not in range(128)
```

So commenting out that line. No idea how that character can appear in the relevant line, it's a £ sign.
This strips out unencodable characters and if not fixes, at least masks #483 by using `codecs.open` and telling it to ignore errors.
@coveralls
Copy link

Coverage Status

Coverage increased (+0.01%) to 85.206% when pulling 1890be2 on issue-483-encoding-errors into 46c1951 on release/2.1.2.

@nigelmegitt
Copy link
Collaborator Author

Whilst this doesn't break anything, it also doesn't seem to solve the problem, whose cause is currently unknown.

@nigelmegitt
Copy link
Collaborator Author

This could be a Python2 issue, to be re-reviewed when we have migrated to Python3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants